Adaptive Regularization Using the Entire Solution Surface A DISSERTATION SUBMITTED TO THE FACULTY OF THE GRADUATE SCHOOL OF THE UNIVERSITY OF MINNESOTA BY

Regularization is essential for obtaining high predictive accuracy and selecting relevant variables in high-dimensional data. Within the framework regularization, several sparseness penalties have been suggested for delivery of good predictive performance in automatic variable selection. All assume that the true model is sparse. In this dissertation, we propose a penalty, a convex combination of the L1and L∞-norms, that adapts to a variety of situations including sparseness and nonsparseness, grouping and nongrouping. The proposed penalty performs grouping and adaptive regularization. In regularization, especially for high-dimensional data analysis, efficient computation of the solutions for all values of tuning parameters is critical for adaptive tuning. In the literature, there exist several algorithms computing an entire solution path for one single tuning parameter. All use the Kuhn-Tucker conditions for constructing the solution path. However, there does not seem to exist such an algorithm for multiple tuning parameters. This is partly because of the difficulty of applying the Kuhn-Tucker conditions when many slack variables are involved. In this dissertation, we introduce a homotopy algorithm utilizing the subdifferential, a systematic method of handling nonsmooth functions, for developing regularization solution surfaces involving multiple tuning parameters. This algorithm is applied to the proposed L1L∞ penalty and permits efficient computation. Numerical experiments are conducted